Combining Perception and Knowledge for Service Robotics

As the deployment of robots is shifting away from the industrial settings towards public and private sectors, the robots will have to get equipped with enough knowl- edge that will let them perceive, comprehend and act skillfully in their new work- ing environments. Unlike having a large degree of controlled environment variables characteristic for e.g. assembly lines, the robots active in shopping stores, museums or households will have to perform open-ended tasks and thus react to unforeseen events, self-monitor their activities, detect failures, recover from them and also learn and continuously update their knowledge. In this thesis we present a set of tools and algorithms for acquisition, interpreta- tion and reasoning about the environment models which enable the robots to act flexibly and skillfully in the afore mentioned environments. In particular our contri- butions beyond the state-of-the-art cover following four topics: a) semantic object maps which are the symbolic representations of indoor environments that robot can query for information, b) two algorithms for interactive segmentation of objects of daily use which enable the robots to recognise and grasp objects more robustly, c) an image point feature-based system for large scale object recognition, and finally, d) a system that combines statistical and logical knowledge for household domains and is able to answer queries such as Which objects are currently missing on a breakfast table? . Common to all contributions is that they are all knowledge-enabled in that they either use robot knowledge bases or ground knowledge structures into the robot s internal structures such as perception streams. Further, in all four cases we exploit the tight interplay between the robot s perceptual, reasoning and action skills which we believe is the key enabler for robots to act in unstructured environments. Most of the theoretical contributions of this thesis have also been implemented on TUM-James and TUM-Rosie robots and demonstrated to the spectators by having them perform various household chores. With those demonstrations we thoroughly validated the properties of the developed systems and showed the impossibility of having such tasks implemented without a knowledge-enabled backbone

Cumulative object categorization in clutter

In this paper we present an approach based on scene- or part-graphs for geometrically categorizing touching and occluded objects. We use additive RGBD feature descriptors and hashing of graph configuration parameters for describing the spatial arrangement of constituent parts. The presented experiments quantify that this method outperforms our earlier part-voting and sliding window classification. We evaluated our approach on cluttered scenes, and by using a 3D dataset containing over 15000 Kinect scans of over 100 objects which were grouped into general geometric categories. Additionally, color, geometric, and combined features were compared for categorization tasks

Wissensbasiertes System für Wahrnehmung der Serviceroboter

As the deployment of robots is shifting away from the industrial settings towards public and private sectors, the robots will have to get equipped with enough knowl- edge that will let them perceive, comprehend and act skillfully in their new work- ing environments. Unlike having a large degree of controlled environment variables characteristic for e.g. assembly lines, the robots active in shopping stores, museums or households will have to perform open-ended tasks and thus react to unforeseen events, self-monitor their activities, detect failures, recover from them and also learn and continuously update their knowledge. In this thesis we present a set of tools and algorithms for acquisition, interpreta- tion and reasoning about the environment models which enable the robots to act flexibly and skillfully in the afore mentioned environments. In particular our contri- butions beyond the state-of-the-art cover following four topics: a) semantic object maps which are the symbolic representations of indoor environments that robot can query for information, b) two algorithms for interactive segmentation of objects of daily use which enable the robots to recognise and grasp objects more robustly, c) an image point feature-based system for large scale object recognition, and finally, d) a system that combines statistical and logical knowledge for household domains and is able to answer queries such as Which objects are currently missing on a breakfast table? . Common to all contributions is that they are all knowledge-enabled in that they either use robot knowledge bases or ground knowledge structures into the robot s internal structures such as perception streams. Further, in all four cases we exploit the tight interplay between the robot s perceptual, reasoning and action skills which we believe is the key enabler for robots to act in unstructured environments. Most of the theoretical contributions of this thesis have also been implemented on TUM-James and TUM-Rosie robots and demonstrated to the spectators by having them perform various household chores. With those demonstrations we thoroughly validated the properties of the developed systems and showed the impossibility of having such tasks implemented without a knowledge-enabled backbone